A study of the universal threshold in the li 
recovery by statistical mechanics 
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Abstract — We discuss the universality of the li recovery 
threshold In compressed sensing. Previous studies In the fields 
of statistical mechanics and random matrix Integration have 
shown that i?i recovery under a random matrix with orthogonal 
symmetry has a universal threshold. This indicates that the 
threshold of ^i recovery under a non-orthogonal random matrix 
differs from the universal one. Taking this into account, we use a 
simple random matrix without orthogonal symmetry, where the 
random entries are not independent, and show analytically that 
the threshold of li recovery for such a matrix does not coincide 
with the universal one. The results of an extensive numerical 
experiment are in good agreement with the analytical results, 
which validates our methodology. Though our analysis is based 
on replica heuristics in statistical mechanics and is not rigorous, 
the findings nevertheless support the fact that the universality of 
the threshold is strongly related to the symmetry of the random 
matrix. 

I. Introduction 

Compressed sensing is nowadays one of the main topics in 
information science, where the sparsity of the signal plays an 
essential role. Compressed sensing has been intensively inves- 
tigated from the theoretical point of view, and its application 
has been attempted in various fields of engineering. 

We start with the basic £i-norm recovery problem proposed 
and analyzed elsewhere H], |J2l, 10. The sensing process in 
this problem is described by a linear equation: 



y = Fx°. 



(1) 



(Bold letters denote vectors and matrices.) x^ e R^ is the 
input signal vector, y e R^ is a P-dimensional observed 
signal vector, and i^" is a P-by-A^ sensing matrix. We assume 
that the entries in F are randomly generated. We define 
compression rate a :— P/N < 1, which is needed for 
the discussion of recovery performance. We also assume that 
input x'^ is also random and that it is drawn from a sparse 
distribution: 



P(x°) = (l-p)<5(xO) + pP(a;?). 



(2) 



Parameter p is the density of non-zero inputs, which is needed 
in the following. The distribution of nonzero entry P{x'^) can 
be set arbitrarily in principle. Here we set it as Gaussian with 
zero mean and unit variance. 
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Within this framework, we consider the 
tion problem with constraint 



-norm minimiza- 



mininiizc \x\ 



subject to y{= Fa;") == Fx, (3) 



where ||a;||p= lime_j.+o X^i l^^il^^*^- In particular we focus on 
the £i problem (p = 1). This equation offers an algorithm 
of recovery for original input a;", which we call £i-norm 
reconstruction. The problem discussed throughout this paper 
is a basic question: in which case does the solution vector of 
^ coincide with original input XqI 

Under the ansatz of random sensing matrix F, the perfor- 
mance of f i-norm recovery has been evaluated using various 
approaches. One study using the restricted isometry property 
[Tl in conjunction with the large deviation theory of random 
matrix spectral edge [3 1 showed that there is a perfect recovery 
region on the (a, p) plane. Another study using analysis of 
random poly tope projection obtained a typical reconstruction 
threshold [4], [5], |_6J that is in excellent agreement with the 
boundary between the success and failure regions obtained in 
an i'l-norm reconstruction experiment. This typical threshold 
(termed weak threshold elsewhere [4|, [51, [6|) can also 
be obtained by statistical mechanical analysis based on the 
replica method f7l, which yields exactly the same analytical 
expression for the recovery threshold. 

Though the rigorousness of the replica method has not 
yet been proven and this method is still a heuristics, it 
has a significant advantage: using the replica method, we 
can analyze problems more general than the basic ^i-norm 
problem. For example, we previously used it to analyze the 
correlated sensing problem JU, ||9]. For other generalizations, 
see the references in ||9l. 

In this article we focus on the universality of the ^i-norm 
recovery threshold discussed in [10|. From the statistical me- 
chanical point of view, this universality can be comprehended 
from rotational symmetry in the matrix integration approach 
as analyzed and elucidated elsewhere |7|, and with this knowl- 
edge we can construct a model that breaks such symmetry. In 
the following we propose a model of symmetry breaking by 
introducing a blockwise-correlated random sensing matrix and 
give an analytical expression for the recovery threshold. Using 
this expression, we can qualitatively trace the deformation 



of the universal recovery threshold by introducing random 
matrix correlation. We also report the results of a numerical 
experiment for which the results are in excellent agreement 
with those of the proposed model. 

This article is organized as follows. First we give an 
overview of statistical mechanical analysis using a replica 
method. Next we address the relationship between this analysis 
and threshold universality from the perspective of matrix inte- 
gration. Then, as an example of how statistical mechanics can 
be used to analyze problems more general than the basic ii- 
norm problem, we use it to investigate the deformed problem 
of an i.i.d. random matrix. We next present a blockwise model 
for observing in detail the deviation from universality. Then 
we describe the numerical experiment we conducted to verify 
the results of the replica method, which lacks rigorousness. 
We conclude with a summary of the key points and a short 
discussion. 

II. Overview of statistical mechanical analysis 

We start with an outline of statistical mechanical analysis, as 
proposed elsewhere [7|. We focus on the basic model, where 
each entry in matrix F is drawn from a Gaussian distribution 
with zero mean and variance N~^. For the moment we do not 
restrict ourselves to the ^i-norm problem and consider instead 
the ^p-norm problem. 

The first step of the analysis is to define quantity Cp: 

1 



Cr, := — lim 



lini 

N^oc 



/3N 



X In 



/ dx exp(-/3 \\x\\p)SiF{x'' - x°)). 



(4) 



This definition describes the minimized £p-norm (divided by 
N) under the condition y{= Fxq) = Fx, which is clearly 
obtained by taking the limit of /3 — > oo. In the present case, 
matrix F and input xq are random, and we need to take 
the average w.rt. them. This requires calculating the average 
of the logarithmic quantity on the rhs, which is an obstacle 
to the analysis. To overcome this obstacle, we resort to the 
replica method, which has not been shown to be rigorous but 
heuristically gives the exact result. With the replica method, 
Cp after averaging is 

nCp]F,^, :=-lim lim ^ lim A lnE[Z"(F, a;°)]p..,o, 

(5) 

where E[ ]x denotes the average w.rt. variable (vector, matrix) 
X; the nth power of factor Z{F, a;") is 

n « 

Z"(F,a;"):= [| dx" e^-^ \\xlp)SiF{x'' - x°)) 

n „ 

= \\ dx'' lim , 

E /3 Ikllp -IT E(^" - xYF^F{x'^ - x^) 



a=l' 



X exp 



a=l 



a=l 



This means that we can estimate the logarithmic quan- 
tity from the positive integer moment with identity 
E[lnX] — lim„^o 51nE[J'('"]/9n. Superscript a on a; 
denotes the "replica" number introduced for estimating 
¥\Z'^{F,x'^)]px«- After the average is taken over Gaussian 
random matrix F and limit t -^ +0, 

E[Z"(F,a;")]ir = f dx" f dQf[ f dx" 

xexpU^TrlnS-E/3|K||Jn(")(Q,a;'^)- (7) 

We omit the trivial overall factor as it is irrelevant to the 
analysis. Square matrix {S)ab '■= Qab — 2Qoa + p is n- 
dimensional; Qab is defined in n(")(<5, a;"). Delta function 
constraint 11*^"' (Q, a;") is given as 

n(")(Q,a;") := 

Y[ / dQaa exp { NQaa{x"^x" ~ NQaa) ] 

dQab exp { NQab{x"^x^ - NQab) \ 
xJI / dQoaexplNQoaix"^x'>-NQoa)\-(8) 

a^l-J-ioo 

In this definition, dual matrix Q is introduced as a collection 
of integration variables for Fourier representation of the delta 
function. 

Computing the n — > limit requires analytic continuation 
from n e N to n G K. To achieve this, we follow the standard 
procedure in the replica method and assume replica symmetry 
regarding matrices Q and Q. Let q — Qab,q — Qab (both 
for a ^ b), Q = Qaa,Q = Qaa, m = Qao and m = Qao, 
yielding Saa = Q — 2m + u and Sab = q — 2m + u {a j^ b). 
By diagonalization of matrix S, Tr In S^ is evaluated as 

TrlnS'= {n- l)ln{Q - q) +Iti{Q - q + n{q - 2m + p)} . 

(9) 

Under the assumption of replica symmetry, the a;"-dependent 
part, namely the ^p-norm and n("'((5, x") are deformed to 

W dx"exp{~Y.(i \\x"\\p n(") (Q, X") 

a=l"^ \ a=l / 



= exp 



— ti(ti — 1) \ 

-NnQQ - N— -qq - Nnmm 



Dz 



i..MN{\Q-\].' 



+a;^(mx° + V^S') -/3|a;|^}])" (10) 



(6) 



where Dz := (a/^tt) ^ f°° dze ^ 1"^ . In the last fine, the 
interaction between replicas (namely x°^ x^') is removed 
by incorporating auxiliary Gaussian variable z (often called 
Hubbard-Stratonovich transformation in physics) and decom- 
posing all replicas. 



From (|9] [TOl i we find that all n-dependent factors are taken 
as defined for n e M (putting mathematical rigorousness 
aside), which allows us to calculate n — ?► limit. For 
convenience of further analysis, we redefine the auxiliary 



variables, m 



/3 ^m,x ■= /3 ^<7, X := PiQ - q), and 



Q :— /3 ^{—2Q + q), and also introduce the function 



0p(/i,Q):=— min^r < — x^ 



hx - 



(11) 




These variables and function are used to simplify the factor in 
/ Dz in ([Toll to cxp{- ^Nn(j)p{mx'^ + y/xz, Q)} for /3 -)■ oo. 
After combining these results and computing the average 
w.r.t. Xq, we compute the limit A^ — > cx). As a result, a six- 
dimensional integral w.r.t. Q, fh, Xj 9: 't^i X is replaced with 
one w.r.t. their extremal values by asymptotic analysis. This 
integral is denoted by the symbol Extr in the following. 
(Although the commutativity of the limits n —>■ and N —>■ oo 
has not been shown, this has not been a concern in standard 
replica analysis.) Finally, after computing the limit n — > 0, we 
arrive at the final expression: 



ncp] 



For evaluation of the threshold, we can extract some informa- 
tion from (fTSl i. As shown in (fT2l i. K[Cp]F,x<:' is nothing but the 
minimized £p-norm after averaging, which can be calculated 
using the extremal values of the six variables. Returning to 
the definition of q, m (delta function in (O) and remembering 
that X is the result of recovery and x'^ is the original input, 
we can see that q ~ m ^ p must be satisfied at the extremal 
of (J, m when the recovery is successful, whereas q ^ m is, 
expected in the unsuccessful case. For threshold evaluation, 
we need to observe the bifurcation from q — ra to q ^ m. 
This statement assumes continuous bifurcation, or second- 
order phase transition in the context of statistical mechanics, 
which is true for the present problem. 

For observing bifurcation, it is more convenient to use the 
variable x{— P{Q ~ l))- In this problem, Q ~ q = m holds 
for successful recovery, while Q ^ q^ m for failure, which 
means x = and x 7^ for success and failure, respectively. 
Henceforth, we use the bifurcation from x = to x 7^ at 
the extremal in (fT2l i. which yields the recovery threshold in 
conjunction with the conditions of the other five variables. For 
•p —\ (£i-norm), the threshold can be expressed as simply two 
equations: 



a = 2{\-p)H\\l^x)+P. 

X = a-i{2(l-p)((x + l)i?(l/Vx) 



-(27r)-i/2y^g-i/2x+p(£+l))|^ (13) 

where il{x) := {2'k)~^/^ J°° dte~* 1"^ is a complementary 
error function (slightly different definition from the standard). 



and elimination of x yields the relation between a and p, 
which is the t\ recovery threshold. This result coincides with 
the weak threshold in [4], [5], [6J, computed from random 
polytope projection. (The equivalence is noted in Q. Actually, 
the extremal condition w.rt. v for cross-polytope in Section 
6.2 in [6| is shown to be the same as ( fTSI l after some algebra.) 

in. Universality from statistical mechanical 
analysis: matrix integration 

Numerical investigation of the universality of the £i-norm 
recovery threshold flOl using several kinds of i.i.d. random en- 
tries in F, such as Gaussian and Bernoulli, and several random 
orthogonal bases, such as Fourier and Hadamard, indicated 
that the threshold under random matrices is universal. 

From a statistical mechanical point of view, this universality 
is understood by the matrix integration formula d?]. Here 
we use the formula from Lie group theory [OH, which is 
equivalent to one from mathematical physics l,12J . called the 
Harish-Chandra-Itzykson-Zuber integral, 

for computing iV -^ 00, where O is an orthogonal, D is 
a diagonal, and L is an arbitrary matrix. (This formula was 
originally given for unitary matrix integration.) All matrices 
are square and A^-dimensional; dO is the Haar measure of the 
A^-dimensional orthogonal group. The function G on the rhs 
is computed from (see e.g. lfT3l . lfT4l . ifTSl ) 

G{x) = ]^j\t(^K{t)--^, (15) 

which is known as i?-transformation in free probability theory 
|:16'|. The function A(i) is implicitly given by Cauchy (or 
Stieltjes) transformation, 

Pd{X) 



dX 



A(x)-A' 



(16) 



where pnW is the density of the diagonal element values in 
D. 

The random matrix ensemble (Wishart ensemble) F'^F, 
where F is a P-by-A^ i.i.d. Gaussian random matrix with 
variance N^^, is assumed to be equivalent to the ensemble 
ODO^ generated by arbitrary orthogonal matrix O under 
the condition that /Or)(A) follows Marcenko-Pastur law ifTTl 
for a = P/N < 1: 

Pd(A) = {l-a)S{X) 

+ ^ ^^^+ ' f^ - ^9(A+ - A)9(A - A_), (17) 

where A± = (1 ± a^^^)^ and Q{x) is a Heaviside function. 
(For a unitary ensemble, the equivalence is as shown in |16|.) 
This is known as the asymptotic eigenvalue density of the 
Wishart random matrix ensemble F^ F. 

We can apply this formula to the average w.rt. random 
matrix F in (|6]l. For ST% . the function G{x) is computed as 
G{x) = —(a/2) ln(l— a;), and applying ( fT4] i to (|6]l (integration 



performed over Haar measure dO) results in an average the 
same as that obtained using ^, as noted in |7|. 

This strongly suggests that rotational invariance of the 
random matrix ensemble F'^F is combined with the universal 
threshold because the result of analysis using matrix integra- 
tion w.rt. the orthogonal group Haar measure is the same 
as that using integration of a Gaussian random matrix with 
i.i.d. entries performed using (|6). This implies that universality 
breakdown requires a random matrix ensemble that breaks 
such symmetry. In the following, we present a symmetry 
breaking model and see how the threshold deviates from the 
universal one. 

IV. Deformed problem 

As mentioned above, statistical mechanics can be used 
to analyze problems more general than the basic ^i-norm 
problem. For example, we previously used it to investigate 
the deformed problem of an i.i.d. random matrix [8], where 
F is given by 

F = /r;h/R^. (18) 

Matrices i?r and Rt are respectively P- and A^-dimensional 
deterministic square symmetric matrices. The square root of 
square matrix A is defined hy A = \fA \fA (e.g. the 
Cholesky decomposition can be used for positive-definite A). 
The H is a P-by-iV rectangular matrix with entries that are 
i.i.d. Gaussian random variables with zero mean and variance 
iV~^. As stated elsewhere |8|, i?r and iit respectively describe 
the correlation among observation vectors and the correlation 
among the representation bases of the sparse input signals. 
(Such a framework is described elsewhere flSl.) When R^ 
and ilt are identities, F becomes an i.i.d. random matrix, and 
the problem returns to the original one. 

We previously applied the replica method to this problem 
and computed E[Cp]ir ^.o |8|. The result was 



E[Cp]f,cdO = Extr 

Q,m,x 



a{q — 2m + u) 
2^ 




+ I n / ^^°^(^°) / Dzcj,p{my/Rtx° + y/^z, Q)\y 

(19) 

where (f)p{h,Q) is given as the solution to an A^-variable 
minimization problem as 

(fp{h,Q):=—mm^ < —x'^ RtX - h^ ^/R'tX+ \\x\\p \ . 

(20) 
The result clearly does not depend on R,. (assuming R,. is 
full-rank), and does only on Rt- This can be understood from 
matrix integration. Suppose that Rt is an identity matrix; the 
ensemble F^F — S'^R^S is then equivalent to ODO^ in 
the previous section because matrix R^ can be eliminated by 
the redefinition of H (after normalization, which changes the 
Marcenko-Pastur law of Wishart ensemble H-'^H and G{x) in 
(fT4] i. however is irrelevant to the universality [8 1). On the other 



hand, Rt cannot be eliminated in the same manner and affects 
the Haar measure dO. This implies that such a random matrix 
ensemble will differ from the one connected from a diagonal 
matrix like Marcenko-Pastur by orthogonal transformation. 

We can calculate the threshold for this deformed problem 
in a manner similar to that used in the previous section by 
investigating the bifurcation from x == to x 7^ 0. Note that, 
in this problem, we must solve an A^-variable minimization 
problem, as in (|20| |. while in the original problem this is simply 
a minimization with only one variable. Such minimization 
generally requires a numerical method (e.g., Monte Carlo) as 
used previously |8|. 

We also studied another deformed problem, where input 
signal x^ is sparse and directly correlated [9|; since this is 
beyond the scope of this article, we omit the details here. 

V. Example of non-universal threshold: 
blockwise model 

For observing in detail the deviation from universality when 
using the deformed model and statistical mechanics both 
analytically and quantitatively, we propose using a blockwise 
model. The sensing matrix in this model is the one given in 
(fTSl l. and Rt is composed of 2-by-2 diagonal blocks. (Input 
size N is assumed to be even). 



-Rt 



accordingly \/Rt — 



1 r 
r 1 



lN/2i 



/_ I 



+ 



<-N/2, 



(21) 



where l± :— (vl + r ± ^/]~^)/2, and In/2 denotes an 
identity matrix of size N/2. yfRt is chosen to be symmetric, 
and Rt — Ip. This blockwise matrix represents the case 
in which all input signals (or equivalently corresponding 
representation bases) are correlated with their partner 

By substituting these signals or bases into ( fT9] l and ( |20] | and 
observing the bifurcation from x = to x 7^ 0, we can obtain 
the recovery threshold for the blockwise model. This analysis 
scheme is almost the same as that previously proposed [81, so 
the details are omitted. 

The final equations for the threshold are 

Dzi I DZ2V {zi,Z2,X) 



X 



tJ"" 



DZ2W(ZI,Z2,X), 



(22) 



which corresponds to (\3\ for the original basic i?i-norm 
problem. The functions V {zi, Z2,x) and W {zi,Z2,x) are 
defined as 

V{zi,Z2,x) 





«i,6=04 








X 


tTi,(T2— ±1 •; 




1 

4(1- 

X Z-i 


r2) 




[ai,a2,zi,Z2,\/x] , (23) 



{(1 - p)%,o + p%,i} {(1 - p)%,o + P%4} 



?i,6=0,l 



E E 

cri,(72— il i,j — 1,2 



4(1 



(^f). 



XX 



^.^V^l , 0-2 , 21 , ^^2 , Vx) ^[^ilea^f^l , 0-2 , 21 , Z2 , Vx) ; (24) 



where i?^ and \/R^ are the 2-by-2 block matrices in (l2Tl i. 
Boolean variables ^i and ^2 represent the case in which each 
input signal is respectively zero and nonzero for pairwise 
input. The functions xi } for each ^1 and ^2 are given as 

Xol{zi,Z2,x) ■= E ^'?l((^+^l +^-^2)VX + 7-??2 -?/lj 

x6 (m Ul-zi + l+Z2)\/x + ri]i -112}] 
+ ^ (1 - r^)nr, [{l+z, + l-Z2)y/^- 77) 

ri=±l 

xe (jT-Zi +T+Z2)\/x + rT] + 1^ 

xe (- {(Lzi +T+z2)Vx + rv - 1}) , 

5(zi,Z2,x) := replace{/+,?_,Z+,Z_} in 

(1) 



„(2) 



(1), 



Xofii^i,Z2,x) with {l_,l+,l_,l+}, (25) 
'^^ r2„n/+zi + l-Z2)\/^ + ra2 - m, 



Xoi{cr2,zi,Z2,x)-^/_^ "»? 
i)=±i 

Eoj(ct2,2;i,22,x) := -™al(o-,2:i,2;2,x) 



„(2)/ 



xi_^(cri,zi,Z2,x):=^ rj 

))=±i 



+ {l-r^){{l-Zl+l+Z2)^/^~<J2}, (26) 
HLzi +/+Z2)yx + rcri -77 



Y^ 



U(cti,zi,Z2,x) := -ra;i,^(cr,zi,Z2,x) 



(2), 



+ (l-r2){(/+zi+/_z2)Vx-'Ti}, (27) 



a;^(CTi,CT2,zi,Z2,x) := (^+zi + LZ2) Vx - ^i + ro-2, 

x[{{ai,a2,zi,Z2,x) •= (Lzi + /+Z2)\/^ - cr2 + rcri, 

(28) 

where ri,,(a;) := .T8(r]a;) for 77 = ±1 and /± := l± — rl^. 
In some cases, x\ } does not depend on cji and/or 0-2 ■ 
Nevertheless, the summations with respect to ai and 0-2 in 
(|23T l and ( l24l l are taken in all cases. 

This is the main result of this article. Summarizing, we 
obtain the analytic expression for the €i-norm recovery thresh- 
old by using the two equations in ( |22] | in a similar form 
as in the original case. The difference is that we need to 
evaluate a double integral in the present problem, whereas that 
in the original case is only single integral (in the definition 
of a complementary error function). When we generalize the 
deformed problem to the it;-blockwise model, we have an 




Fig. 1. £i-norm recovery threshold as function of p and r. Areas above 
and below surface ai'e success and failure regions, respectively. Thick curve 
at r = 1 coiTesponds to threshold at r = (i.e. universal threshold), which 
is drawn for comparison and for illustrating deviation. In region of large r, a 
slight deviation from universality is evident. 




Fig. 2. Dependence of recovery thi'eshold on correlation parameter r for 
p = 0.5. Horizontal line corresponds to universal threshold value at p = 0.5. 
For large r, deviation from universality is apparent. 



expression of the recovery threshold equations with a w-tuple 
integral. The problem we previously dealt with [8 1 corresponds 
to the case in which w = N, which requires a Monte Carlo 
approach to the evaluation of multiple integrals. 

Using this analytical expression of the recovery threshold 
for the deformed problem, we consider the deviation from the 
universal threshold. The dependence of recovery threshold a 
on two parameters (p and r) is depicted in Fig. [T] In the region 
where r sa 1, a slight deviation from the universal threshold 
is evident. The deviation is shown in detail in Fig. |2] where 
p = 0.5. In the region of a larger r, a clear deviation from the 
universal threshold (shown by the horizontal line) is evident. 

VI. Validation by numerical experiment 

Given that the replica method lacks rigorousness, we ver- 
ified its results by conducting a numerical experiment of 
£1 recovery. We used the convex optimization package for 
MATLAB fT9), f20l and evaluated the recovery threshold. We 
first prepared a square random sensing matrix F with size N 
and deleted the rows in F one-by-one until recovery failure 
occurred, as determined using |a;* — x^\ > 10^'' for recovery 
result cc,. We recorded the number of remaining rows (plus 
one) Pc = P + 1 at the point of failure. We repeated this 
10'^ times and then computed the arithmetic average of Pc/N, 




Fig. 3. Result of £i-norm recovery experiment for p = 0.5 and r = 0.9. 
Number of input signals A'^ was varied from 20 to 110 in steps of 10. For 
each N, the average was taken over 10^ samples. Broken curve indicates 
scaling by quadratic function regression. Extrapolated value at A'^ — > oo was 
0.8370(3), while result of statistical mechanics evaluated by A'^ — > oo limit 
(horizontal line) was 0.83649.... These results are in excellent agreement. 



which we regarded as the value of a at the recovery threshold. 
The results are plotted in Fig. |3] in which the dependence 
of the threshold value on the dimension of input signal TV is 
depicted. We also performed scaling analysis using quadratic 
function regression and estimated the value of the threshold for 
N —^ oo limit. The value of the threshold from extrapolation 
was 0.8370(3) for A^ ^ oo. This value is in excellent 
agreement with the result obtained using statistical mechanics 
(0.83649...), which validates our analysis for the deformed 
problem. 

VII. Summary and Discussion 

We presented a deformed model for ^i-norm reconstruction, 
that is, a blockwise correlation model that represents the 
pairwise correlation in signals. From this model we obtained 
an analytical expression for the .^i-norm recovery threshold by 
replica heuristics. Using this expression with a double integral, 
we evaluated the threshold and found a clear deviation from 
the universal threshold in the region of strong correlation. A 
numerical experiment validated the results obtained with this 
model. This model enables minute deviations from the univer- 
sality of the ti recovery threshold to be traced qualitatively 
by using an analytical expression of the threshold. 

We showed that this blockwise model yields a non-universal 
threshold, as expected from the rotational symmetry breaking 
argument. This suggests that orthogonality of the representa- 
tion bases in sensing matrix construction (see [IS]) is crucial 
for universality. The importance of orthogonality to univer- 
sality was investigated by other researchers in terms of the 
restricted isometry property llTSl . (1211 . and our results support 
their findings. 

The relationship of our analysis to random polytope projec- 
tion is of great interest. As we noted in Section |II] for the orig- 
inal model, both geometrical and statistical mechanical anal- 
yses give the same threshold. For the deformed problem, like 
the one we handled using a blockwise model, the rotational 
symmetry is broken. It thus appears that a "biased" projection 
should be taken into account in geometrical analysis, where 



projection group symmetry, a quotient of rotational group, 
is generally assumed. The results presented here should be 
useful for obtaining a deeper understanding of the relationship 
between geometrical and statistical mechanical analyses. 

Acknowledgments 

This research was supported by KAKENHI (Nos. 22300003 
and 22300098) and the Mitsubishi Foundation (YK). 

References 

[I] E. J. Candes, J. Romberg, and T. Tao, "Robust uncertainty principles: Ex- 
act signal reconstruction from highly incomplete frequency information," 
IEEE Trans. Inform. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006. 

[2] D. L. Donoho, "Compressed sensing," IEEE Trans. Itrform. Theory, vol. 

52, no. 4, pp. 1289-1306, Apr. 2006. 
[3] E. J. Candes and T. Tao, "Near optimal signal recovery from random 

projections: Universal encoding strategies?," IEEE Trans. Inform. Theory, 

vol. 52, no. 12, pp. 5406-5425, Dec. 2006. 
[4] D. L. Donoho and J. Tanner, "Neighborliness of randomly projected 

simplices in high dimensions," Proc. Natl. Acad. Soc, vol. 102, no. 27, 

pp. 9452-9457, Jul. 2005. 
[5] D. L. Donoho, "High-Dimensional Centrally Symmetric Polytopes with 

Neighborliness Proportional to Dimension," Discrete Comput. Geom., 

vol. 35, no. 4, pp. 617-652, May 2006. 
[6] D. L. Donoho and J. Tanner, "Counting faces of randomly projector 

polytopes when the projection radically lowers dimension," J. Amer. 

Math. Soc, vol. 22, no. 1, pp. 1-53, Jan. 2009. 
[7] Y. Kabashima, T. Wadayama, and T. Tanaka, "A typical reconstruction 

limit for compressed sensing based on Lp-norm minimization," J. Stat. 

Mech., L09003, Sep. 2009. 
[8] K. Takeda and Y. Kabashima, "Statistical Mechanical Analysis of Com- 
pressed Sensing Utilizing Correlated Compression Matrix," Proc. 2010 

IEEE Int. Symp. Inf. Theory, pp. 1538-1542, Jun. 2010. 
[9] K. Takeda and Y Kabashima, "Statistical Mechanical Assessment of 

a Reconstruction Limit of Compressed Sensing: Toward Theoretical 

Analysis of CoiTelated Signals," Europhys. Lett., vol. 95, no. 1, 18006, 

Jul 2011. 
[10] D. L. Donoho and J. Tanner, "Observed universality of phase transitions 

in high-dimensional geometry, with implications for modern data analysis 

and signal processing," Phil. Trans. R. Soc. A, vol. 367, no. 1906, pp. 

4273-4293, Nov 2009. 

[II] Harish-Chandra, "Differential operators on a semisimple Lie algebra," 
Amer. J. Math., vol. 79, no. 1, pp. 87-120, Jan. 1957. 

[12] C. Itzykson and J. -B. Zuber, "The planar approximation II," J. Math. 

Phys., vol. 21, no. 3, pp. 411-421, Mar. 1980. 
[13] E. Marinaii, G. Parisi, and F. Ritort, "RepHca field theory for detennin- 

istic models: II. A non-random spin glass with glassy behaviour," J. Phys. 

A, vol. 27, no. 23, pp. 7647-7668, Dec. 1994. 
[14] R. Cherrier, D. S. Dean, and A. Lefevre, "Role of the interaction matrix 

in mean-field spin glass models," Phys. Rev. E, vol. 67, no. 4, 046112, 

Apl. 2003. 
[15] K. Takeda, S. Uda, and Y Kabashima, "Analysis of CDMA systems that 

are characterized by eigenvalue spectrum," Europhys. Lett., vol. 76, no. 

6, pp. 1193-1199, Dec. 2006. 
[16] A. M. Tuhno and S. Verdii, Random Matrix Theory and Wireless 

Communications, Hanover USA: Now publishers, 2004. 
[17] V. A. Marcenko and L. A. Pastur, "Distributions of eigenvalues for some 

sets of random matrices," Math. USSR-sbomik, vol. 1, no. 4, pp. 457-483, 

Apl. 1967. 
[18] E. J. Candes and M. B. Wakin. "An Introduction To Compressive 

Sampling," IEEE Signal Process. Mag., vol. 25, no. 2, pp. 21-30, Mar. 

2008. 
[19] M. C. Grant and S. P Boyd, CVX: Matlab software for 

disciplined convex programming (web page and softwai'e) 2009. 

http://stanford.edu/~boyd/cvx 
[20] M. C. Grant and S. P. Boyd, "Graph implementations for nonsmooth 

convex programs," Recent Advances in Learning and Control, V. Blondel, 

S. Boyd and H. Kimura eds., pp. 95-110, London: Springer- Verlag, 2008. 
[21] R. Baranuik. M. Davenport, R. DeVore, and M. Wakin, "A simple proof 

of the restricted isometry property for random matrices," Constr. Approx., 

vol. 28, no. 3, pp. 253-263, Jan. 2008. 



